TransFGU: A Top-Down Approach to Fine-Grained Unsupervised Semantic Segmentation

AbstractUnsupervised semantic segmentation aims to obtain high-level representation on low-level visual features without manual annotations. Most existing methods are bottom-up approaches that try group pixels into regions based their cues or certain predefined rules. As a result, it is difficult for these generate fine-grained when coming complicated scenes with multiple objects and some sharing similar appearance. In contrast, we propose the first top-down unsupervised framework in extremely scenarios. Specifically, rich structured concept information from large-scale vision data self-supervised learning manner, use such as prior discover potential categories presented target datasets. Secondly, discovered mapped pixel by calculating class activate map (CAM) respect representation. Lastly, obtained CAMs serve pseudo labels train module produce final segmentation. Experimental results benchmarks show our robust both object-centric scene-centric datasets under different granularity levels, outperforms all current state-of-the-art methods. Our code available at https://github.com/damo-cv/TransFGU.